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AN URBAN FAMILY MATH COLLABORATIVE 



FINAL REPORT 
Executive Summary 

This project measured the impact of a short series of Family Math programs in elementary grades 
on student and parent attitudes toward mathematics, student performance in mathematics, and 
teacher behavior using control-experimental comparison groups and supplemented by parent and 
teacher interviews. Only two of the analyses showed statistical significance. Students in the 
experimental group who had prior Family Math experience showed higher gains on standardized 
mathematics performance assessment measures than other groups. Also, parents who attended 
Family Math reported increased involvement with their childrens' schools. Parent interviews 
indicated strongly favorable attitudes toward Family Math and had high praise for the quality of the 
program, although these results were not supported by analysis of the parent attitude questionnaires. 
Teacher interviews indicated enthusiastic support for Family Math and substantial modification of 
teaching methods as well as improved understandings about mathematics. In view of the limited 
extent of the project as well as the offering of three sessions per semester over three semesters and 
the finding that fewer than one-quarter of the students attended all three sessions in each semester, 
the positive outcomes in student performance, parent increased involvement in schools, and parent 
and teacher interview responses are indications that Family Math appears to impact the participants. 
Further study is recommended. A number of new Family Math teachers and staff developers were 
also trained and certified during this project and newly developed materials for upper elementary 
and middle school grades were disseminated for field testing. 

Introduction 

Family Math is a well-documented program in which parents and their children attend a series of 3 
or more 2 hour weekly sessions, usually after school, in evenings or on weekends. The focus of 
Family Math's hands-on, enjoyable activities is on conceptual development and strategies related to 
mathematics rather than single answer problems. The program not only aims to shape students' 
attitudes toward math, related studies, and careers, but also expects to influence parents to reinforce 
their childrens' interest in math. 

This project sought to determine whether attendance at a succession of Family Math programs 
during the elementary grades would have a measurable impact on student and parent attitudes, 
student performance in math and teacher behavior. In addition, new Family Math materials for 
grades 5 through 8 created by Virginia Thompson, Director of Family Math/EQUALS at the 
Lawrence Hall of Science in Berkeley, California were field tested. During the course of the 
project a number of new Family Math teachers and staff developers were trained and certified. 

Community School District #15 was selected as the operating site for the project because of the 
cooperation of William P. Casey, the Community Superintendent, the dynamic leadership of 
Josephine Urso, the District Math Coordinator, and the large number of trained Family Math 
teachers. District #15 in northwest Brooklyn is part of the New York City Public School System 
and is described as working class and culturally, linguistically and racially diverse. Josephine Urso, 
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as Senior Staff Developer, is the only person in New York who is authorized by Virginia Thompson 
to train and certify new Family Math staff developers. 

The Center for Advanced Study in Education (CASE) of the City University of New York (CUNY) 
Graduate School is the National Family Math Site for New York City. The Site is coordinated b> 
Dr. Stan Brodsky. CASE has facilitated the training of 34 staff developers and more than 450 
Family Math teachers. Two highly qualified project evaluators, Drs. Marian C. Fish and Alan 
Gross, are members of the Graduate Educational Psychology Faculty at the CUNY Graduate School. 



Process 

Teacher Orientation and Training 

An initial orientation meeting with key teachers from District #15 was held on March 17, 19^3 to 
describe the research and project procedures. An agenda for this meeting and an outline of the 
presentation about the Family Math project are included in Appendix A. 

Virginia Thompson and her colleague, Karen Mayfield, scheduled a visit to District #15 on January 
28, 1994. In December, an invitation to attend the morning January 28 meeting with Virginia 
Thompson was sent to all District Math Coordinators in the City, to all Family Math Staff 
Developers and a selection of others. The purpose of that meeting was to introduce them to the 
newly developed Family Math activities for 5th through 8th grade which the participants were asked 
to help to field test. Those districts that were unable to be represented were also sent the new 
materials for use by their Family Math teachers. 

On the afternoon of January 28, 1994 a meeting of District #15 Family Math teachers with Virginia 
Thompson and Karen Mayfield was held. Twenty three Family Math teachers from the district 
attended. The project procedures. #ere reviewed and the new Family Math materials were 
disseminated. Several other Family Math teachers from participating schools who could not attend, 
were briefed individually by Josephine Urso. 

Another feature of this project was the preparation of additional teachers of Family Math both in 
District #15 and other districts across the City as well as a small group of Family Math staff 
developers to continue the expansion of the Family Math teaching corps. Workshops were 
scheduled for July 19 & 20, 1994 for new teacher preparation, which required 12 hours, and for 
July 18, 19 & 20, 1994 for preparing a group of experienced Family Math teachers to become staff 
developers, which required 18 hours. A total of 9 new Family Math teachers were trained for 
District #15, 28 were trained for other New York City Public Schools entities, and one from out of 
New York City was trained; a total of 38 new Family Math teachers. In addition, 4 new Family 
Math staff developers were qualified. 

Current Prefect 

This Family Math program was conducted over an 18 month period, covering three academic 
semesters. The initial period was the Spring 1993 semester for which data are indicated as 1993. 
The second period, consisting of two semesters which comprise a full academic year, Fall 1993 and 
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Spring 1994, is represented by data identified as 1994. During the Spring 1993 period, families 
with fourth, fifth and sixth grade students from 10 elementary schools were offered three 
consecutive Family Math sessions. The 1994 cohort was drawn from 1 1 elementary and 1 middle 
school and included a small number of seventh graders. The same families who attended Family 
Math as part of the 1993 groups were encouraged to attend the Family Math sessions in 1994. In 
addition, new families were invited to participate in the 1994 groups at all grade levels. 

Evaluation Plan 

Four evaluation instruments were devised by the evaluators, tested and suitably modified. These 
instruments were a Student Background Questionnaire to be completed by the teacher, a 10-item 
Student Attitude Questionnaire, a Parent Questionnaire in both English and Spanish, and a Parent 
Family Math Evaluation Form. The Student Attitude and the Parent Questionnaires were pre and 
post measures. The first three instruments were expected to be completed for both the Family Math 
(experimental) group and the control group. Both parent forms and the Student Attitude 
Questionnaire were organized in checklist formats. Copies of these forms appear in Appendix B. 

Student conceptual math ability was assessed using materials prepared and validated by CTB 
Macmillan/McGraw-Hill through the cooperation of Dr. Nick Maruhnich. Copies of these 
instruments are not included because the condition under which they were made available to this 
project required that they be secure and confidential. These materials were designed to reflect the 
latest National Council of Teachers of Mathematics standards for both content and process outcomes 
which are the areas that Family Math would most likely influence including concept and strategy 
development and problem-solving approaches. The assessment consisted of open-ended, constructed 
response tasks. 

In addition to the measures noted above, student performance on norm-referenced standardized 
achievement tests used by the New York City Public Schools was included. An SPSS computer 
program was designed and modified by Dr. Alan Gross to analyze the available data. 

The evaluetors also prepared a semi-structured format for interviewing a sample of the Family Math 
teachers and a second such format for interviews with a sample of parents who attended Family 
Math sessions with their children. These interviews were carried out during the Spring 1994 
semester. 

One major question addressed by this evaluation was whether students who participated in Family 
Math showed higher gains on mathematics performance measures (i.e. standardized test scores and 
performance assessment measures) than comparable students who did not participate in the program. 
Additional questions addressed include: 

a) did student participants show more favorable attitudes toward mathematics than students 
who did not participate? 

b) did parent participants show more favorable attitudes toward mathematics than parents 
who did not participate? 

c) did participation in the program have an impact on family involvement with the school? 

d) how did parents and students evaluate the Family Math sessions? 

e) in what ways did teaching Family Math affect teacher behavior? 



Evaluation Methodology 



The major question of program impact on standardized test scores and mathematical performance 
assessment measures was assessed using a control/experimental group design. The experimental 
group consisted of those students and parents who participated in the Family Math program. Within 
each school, a control group was randomly selected from school records. In 1993, there were 101 
students (and 101 parents) in the experimental group and 89 students in the control group. In 1994, 
there were 211 students in the experimental group and 234 students in the control group. Initial 
differences between groups were statistically adjusted using analysis of covariance to compare post- 
score differences between control and experimental groups. In analyzing program effects, the 
experimental group was broken down into two groups, those having had prior Family Math 
experience and those who did not. Thus, all statistical comparisons involved three groups,; the 
control group, and the two experimental groups. It should be noted that permission of the New 
York City Board of Education was required to obtain the test score data for all students. Student 
Background Questionnaires and attendance sheets were filled out for each student by the Family 
Math teachers. 

To analyze student attitudes, the Student Attitude Questionnaire was developed. Modeled after 
standard measures (e.g. the Dutton Scale), it was composed of 10 items that asked students to 
describe their attitudes about math by responding "never", "some of the time", "most of the time", 
or "always" to each item (e.g. "I enjoy doing math"). The student attitude data was analyzed using 
the same control/experimental group design described above. Both control and experimental 
students completed attitude questionnaires, pre- and post-participation. 

To analyze parent attitudes, the Parent Questionnaire was developed. The original questionnaire 
consisted of 25 items that assessed three areas: parent self-confidence with respect to math, parents' 
perception of their child's self-confidence with respect to math, and family involvement with school 
activities. Due to concern about its length, this questionnaire was shortened to 16 items for the 
1994 groups. Parents were asked to respond "strongly disagree", "mildly disagree", "mildly 
agree", and "strongly agree" to each item (e.g. "Math is one of my favorite subjects"). The parent 
data was analyzed using the same control/experimental groups described above. Both control and 
experimental parents completed questionnaires pre- and post-participation. Copies of the three 
questionnaires noted above and the evaluation form mentioned in the following paragraph are shown 
in Appendix B. 

The Parent Family Math Evaluation Form was administered to parents at the end of each Family 
Math series asking them to rate the "overall quality of the sessions" and the "everyday usefulness of 
the sessions", what they had accomplished, whether they would "like to attend more Family Math 
classes next year" and whether they would "recommend Family Math to a friend". In addition, 10 
parents who participated in the Family Math workshops during the Fall 1993 semester were 
interviewed in the Spring 1994 to assess parent reactions to Family Math and to determine in what 
ways Family Math affected parent or child attitudes or behavior. 

Finally, a sample of 10 Family Math teachers were interviewed to assess the impact of the program 
on their attitudes concerning mathematics education and instructional practices. 
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Results 



Background Data 

The background data for the 1993 and 1994 cohorts are presented in Table 1 separately for the 
control and combined experimental groups. The results in Table 1 can be summarized as follows: 

a) In both cohorts, students were primarily in grades 4 through 6. 

b) For both cohorts there were slightly more female than male students in both the control 
and experimental groups. 

c) The range for* the percentage of Limited English Proficiency (LEP) students was from 
6.5% for the 1994 control group to 16.8% for the 1993 experimental group. 

d) The majority of the students participated in the free lunch program. 

e) In both cohorts, the students in the combined experimental groups had attended Family 
Math sessions prior to the current study more often than those students in the control group. 

f) Most students did not attend all three Family Math sessions offered in either cohort. In 
1993, 61% of the students attended two or more sessions. In 1994, the percentage was 64%. 

g) The percentage of parents attending decreased between session 1 and session 3. 

Attitude Data 

Table 2 summarizes the responses to the Parent Family Math Evaluation Form which provides 
parent ratings of the Family Math program as described above. Average parent ratings of the 
overall quality of the sessions, presentation of materials and the everyday usefulness of the sessions 
were consistently in the "Excellent" category. On a scale of 1 ("Excellent") to 5 ("Poor") the 
average ratings ranged from 1.3 to 1.5. More than two-thirds of the parents reported having 
accomplished the following goals: 

a) helping their child with math 

b) providing math materials at home 

c) knowing more about math 

d) learning how math is taught in school. 

Over 90 % of the parents stated that they would like to attend more Family Math classes next year 
and would like to recommend the program to a friend. 

The Family Math Parent Interview data strongly supported these positive ratings. As discussed in a 
subsequent section, parent enthusiasm for Family Math was evident through their desire to share it 
with others and to expand the program hours. 

Comparisons of the Control and Two Experimental Groups 

In Table 3 the effect of participating in Family Math was analyzed using a series of analyses of 
covariance. Three different groups were contrasted: a control group (C), an experimental group of 
students who had had no prior Family Math experience (NPFM) and an experimental group of 
students who had prior Family Math experience (PFM). These groups were compared on the 
following set of post-scores: 
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a) score on a mathematics performance assessment measure (MATH), an instrument which 
measures conceptual math ability, that was prepared and validated by CTB Macmillan/McGraw-Hill 
(in collaboration with the Maryland State Department of Education, 

b) the national percentile ranking (NP) on the California Achievement Test, a standardized 
achievement test administered annually by the New York City Board of Education, 

c) the Student Attitude Questionnaire, comprised of 10 items which measure student attitudes 
toward math, 

d) a measure of parent self-confidence with respect to math; this score was the average of 
specific itens on the Parent Questionnaire and is denoted as PARI, 

e) a measure of parents' perception nf their child's self-confidence with respect to math; this 
score was the average of specific items on the Parent Questionnaire and is denoted as PAR2, and 

f) a measure of parent involvement with school activities; this score was the average of 
specific items on the Parent Questionnaire and is denoted as PAR3. 

In comparing the postscores of the three groups, the corresponding prescores served as the 
covariates except for the post NP measure, where pre-math was the covariate. 

In Table 3 the adjusted mean scores are reported for each of the three groups for each of the 
postscore measures for the 1993 and 1994 cohorts. Two significant results were obtained. First, in 
Spring 1994 the PFM (prior Family Math) group scores significantly higher on the NP (standardized 
test measure) than the other two groups. The average percentile score for the PFM group was 
approximately 11 points higher than the NPFM experimental group. Second, in 1994 both 
experimental groups scored higher on the PAR3 measure (parent involvement with school activities) 
than the control group. Whereas the mean ratings for the two experimental groups were 2.8 and 
2.9, the mean rating for the control group was 2.3. No other results were statistically significant. 

Finally, the Family Math Teacher Interview data, which is reported in detail below, revealed that 
teachers were very enthusiastic about Family Math and that leading Famil Math workshops 
broadened their own ideas about teaching math. They observed increasing self-confidence on the 
part of students and parents with regard to math. 

Teacher Interviews 

Purpose , A series of interviews with teachers who have led Family Math workshops during the 
Spring or Fall 1993 were conducted during late February and early March 1994. The purpose of 
these interviews was to assess teacher reactions to Family Math and to determine in what ways 
teaching Family Math affects teacher attitudes and behavior. 

Background , In Spring 1994 this Family Math project was running in 12 of the schools in grades 
4, 5, 6 and 7. Ten teachers from District #15 were interviewed, eight at their work sites, and two 
on the telephone, using a semi-structured interview format. Seven teachers were female and three 
were male. The teachers were from six different schools, either pre-K to 6th grade or kindergarten 
to 6th grade; one teacher worked at the district office. This is an experienced group of 
professionals as all but one teacher had been teaching for 12 or more years. The teachers varied in 
the number of times they had led Family Math workshops. About half were fairly new leaders with 
between one and five workshops completed, while the other half had led the workshops between 10 
and 16 times. Most had either co-lead the Family Math workshops or had worked with an assistant. 



Family Math workshops were generally held right after school, from about 3:15 to 5:15, although 
two teachers held them in the evening from about 6:00 to 8:00. 



Analysis of Interview Data . Why did you decide to get involved in Family Math? In general, the 
teachers involved in Family Math described themselves as enjoying math; they wanted to pursue 
further activities related to math. Some were asked by their principals or other administrators to 
participate; others had tried writing curriculum or booklets and wanted a change; all were curious 
about the program. Josephine Urso, the District Math Coordinator, generated enthusiasm about the 
program as well. The philosophy of the program, joining parent and child and getting parents more 
involved in schools, appealed to a number of them. One teacher, an exception, said she had "math 
anxiety" and wanted to participate to help overcome her fears. 

Has Family Math changed your attitude toward math or your method or style of teaching math in 
the classroom? Teacher attitudes toward math continue to be very positive after participation in 
Family Math. Teachers reported that it "validated my feelings toward math," "gave me a better 
understanding of math," and "made me think differently about how to teach math." About half of 
the teachers said they changed their own classroom practices by "doing a lot more hands on 
activities," "greater emphasis on talking through math problems," "more development of concepts," 
and "more manipulative activities." The others said it reinforced what they were already doing; one 
teacher said it promoted interests in different areas for her including math through literature. 

What concerns have arisen in implementing the Family Math program? Concerns raised fall into 
two categories: preparation time and workshop issues. Preparation time . Substantial preparation 
was required in order to be sure that the workshop ran smoothly according to most of the teachers. 
Gathering materials, planning activities and developing new activities for participants who had 
attended prior workshops were time consuming. Most of the teachers prepared during school time 
when they had "prep" periods or during lunch. Several teachers reported that the preparation time 
was reduced as they gave more workshops because they had their materials ready; however, two 
teach rs who have led numerous workshops and do not want to repeat activities, are using their time 
to search for new ones to maintain interest. One teacher said "the workshop is the easy part." All 
of the teachers were very frustrated by the additional paperwork they were required to complete 
because of the grant evaluation. They said that this takes up time and energy that would best be 
used for the workshop. Also, they are concerned that this "turns off" participants. Workshop 
issues. The main issue that arose regarding the workshop was regulating the flow of people. In 
several schools the turnout was so high that the teacher felt "pulled all over" as she tried to help 
everyone at once. Other teachers were disappointed when only half of those expected showed up. 
Some teachers saw a drop off in numbers over the three sessions; others did not. It seems that the 
inability to predict the number of participants is somewhat disconcerting. A few issues arose during 
the workshop. For example, some parents brought younger siblings who were occasionally 
distracting or disruptive. Another teacher mentioned that in her sessions the parents got "chatty" 
(with each other) and the children wanted their attention back. One teacher was initially anxious 
about having parents and colleagues present, but once she got used to it, she enjoyed it immensely. 

How did you handle non-English speaking participants? Several teachers had participants whose 
primary language was not English. Written materials were available in Spanish, and one teacher 
made duplicate sets for all sessions. She also asked a bilingual parent to assist her. Another teacher 
who was bilingual chose not to give out the Spanish materials and switched to Spanish when it was 
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necessary. She found that even non-English speaking parents had little difficulty understanding 
because of the "hands on" nature of the workshop and the modeling by the leader. For example, 
there was no translator for some Chinese participants, and they followed along without a problem. 

What kinds of assistance have you received when you needed help? In general, the workshops 
went smoothly and the teachers enjoyed doing them. They were always able to rely on Josephine 
Urso for assistance; in particular, when they gave their first workshop, she offered materials and 
came to the workshops to help or sent someone else. As they became more experienced, the 
teachers began to rely more on each other for sharing ideas and experiences. 

What changes did you make while teaching Family Math? Most of the teachers reported closely 
following the activities in the book. Some of the more experienced Family Math, teachers developed 
some of their own activities, and some teachers reported occasionally changing the materials. 
Variations on activities to simplify concepts also occurred. 

How would you change Family Math to work better? As might be expected, either eliminating 
the paperwork (particularly from the grant evaluation) or having someone assist with the clerical 
tasks was an oft-repeated suggestion. Also, providing a pool of materials, already prepared, and 
more new activities was a frequent response. One teacher suggested getting paraprofessionals who 
work in the classrooms with Family Math teachers involved in the project so that they could prepare 
materials together and benefit from the exposure to the program. One teacher would like to provide 
a book or some other relevant reward to students who attend all three sessions. She saw this as a 
way of reinforcing students, but also of conveying the concept of math through literature. Another 
teacher suggested having more than three workshop sessions each semester. She felt that the 
participants wanted to continue coming and that there was too much time in between the fall and 
rpring sessions. Finally, one teacher would have liked some more training about how to go over 
answers. He suggested that new Family Math leaders serve as "apprentices" or assistants to more 
experienced leaders; he also suggested having a training video or slides to "walk through" the 
sessions. Interestingly, he was originally a science teacher while all of the others were math 
teachers. 

What observations have you made about the impact of the Family Math program on the parents? 

The teachers were delighted with the response of the parents. First their attitudes toward math 
seemed to become more positive. Some parents were initially afraid of math; this changed to "I can 
do that. " Teachers felt that parents became more confident and even competitive. Second, the 
parents enjoyed themselves. "They had fun and verbalized it, laughing and chatting." Parents met 
other parents and became friendly. Third, they saw parents working with their children in a 
cooperative way, talking through problems. "Seeing parents and children working together for two 
hours without arguing is great." They are hopeful that this improves the parent-child relationship. 
A few teachers mentioned that parents became friendlier with the teacher and more active in the 
school as well. 

What observations have you made about the impact of the Family Math program on the students? 

The consensus was that the appeal of math was increased for students who attended Family Math. 
This was said in many different ways: "They enjoy math a lot more," "I have to throw them out at 
the end of the session or they won't leave," and "They get excited when the Family Math letters are 
going out." Some brought older brothers and sisters to the sessions. Others discussed Family Math 
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in the classroom. Teachers reported that the students became more confident and relaxed; they 
observed that students tended to think through problems more often. It was noted that the students 
in Family Math for this grant were in grades 4, 5 and 6 rather than the primary grades. This is 
often a "tougher" audience to please, but it appears as though there was great motivation. 

Summary , The teacher response to Family Math was very enthusiastic. They expanded their own 
ideas about teaching math, conscientiously gathered and prepared materials, and modified existing 
activities where appropriate. They were supported and encouraged by both Josephine Urso and their 
colleagues. They were rewarded by seeing parents and students who enjoyed the sessions 
enormously and whose attitudes toward math became more positive. They saw increasing self- 
confidence on the part of parents and students with regard to math, and they delighted in the 
cooperation between parent and child as they worked together. 

Parent Interviews 

Purpose , A small group of parents who participated in the Family Math workshops during the Fall 
1993 were interviewed in the Spring 1994. The purpose of the interviews was to assess parent 
reactions to Family Math and to determine in what ways Family Math affected parent or child 
attitudes or behavior. 

Background . Ten participants, 8 mothers, 1 grandmother and 1 father who attended Family Math 
in the fall were interviewed by telephone for this evaluation. Four of the parents had attended one 
Family Math workshop (Fall 1993), while the others had attended 2, 3, 4 or 6 times in prior years 
(often with their older children). Parents interviewed were from four different schools in the 
district. Four of the parents did not speak English as a first language. Five of the parents worked 
full-time. Several participated in other school activities, but most did not. 

While all the parents who attended Family Math had a child in either 5th or 6th grade, over half of 
the parents brought siblings to the workshops as well. The siblings ranged from 3 to 15 years old. 
Also, in this group there were two couples who attended with their children. 

Analysis of Interview Data , Why did you decide to attend Family Math? Several reasons for 
participation were mentioned. First, parents said that they thought it would be enjoyable to go to 
Family Math and that, in particular, it was fun to do things with their children; they stressed over 
and over the importance of the "family" aspect. The father, who often worked the night shift, said 
he went because "I hardly have time to do things with my daughter." A second reason was that the 
child asked or encouraged the parent to attend. One mother was reluctant to go, but her daughter 
signed them up. Third, several parents mentioned that they thought it would benefit their children 
because "math is Jane's weakest area," or, "if I go, my grandson will be more interested in math." 

Did your participation meet your expectations? Without exception, all of the parents felt that the 
Family Math workshops had met their expectations. Comments included "I had fun," "I learned a 
lot," and "met my expectations and more." 

Has your attitude toward math changed in any way? All but two of the parents answered "yes" 
to this question. About half indicated that they had learned math differently or were "scared" of 
math before going to Family Math. One woman said, "when I was young, I didn't like math so 
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much in my country, and I was confused. Now, I'm studying too... math is fun and interesting." 
Another said, "It helped me to realize that math could be fun." Of the two parents who did not 
report a change in attitude, one always liked math (and still does), and the other always disliked 
math, found it hard (and still does). 

Has your relationship with your child (with regard to math) changed in any way? All but one 

participant reported changes in the child-parent relationship with regard to math. In general, they 
agreed that it was now more enjoyable to do math together and that the children were less hesitant 
to ask for help. Over half of those interviewed mentioned that they tried to incorporate math 
activities more often into daily activities, for example, when cooking and shopping. One mother 
said, "It brought us closer together, even if only for math." The father who reported no change 
indicated that his wife was the one who spent time on math with his daughter because of his evening 
work hours. 

What observations have your made about your child and math? The parents report that the 
children were much more enthusiastic about math after participating in the Family Math workshops. 
They described their children as enjoying math more and finding math more interesting. One 
mother found that "the pressure is off, and my daughter is more relaxed about math." Several 
parents said their children liked to explain their math homework to them. 

Has your participation in (other) school activities changed in any way? In general, there was no 
reported change in parents' school-related activities. About half of the parents are already active in 
the school either as volunteers or on the PTA or, in 2 cases, as teachers. They remain active and 
Family Math is just one of many activities they attend. For the other five parents, Family Math is 
the only school activity in which they have participated. They were clear that it was the "family" 
aspect that made Family Math different from other school activities and that appealed to them. 

Has your relationship with your child's teacher changed in any way? Once again about half of 
the parents did report changes in relationships with the teachers as a result of attending Family 
Math. Most of the comments reflected greater confidence in approaching teachers and the school in 
general. One parent commented that she is more likely to call the teacher's hot line or send a note 
if she doesn't understand something, whereas before "I would have been embarrassed." Another 
parent felt that she got to know more teachers in the school which made her more comfortable when 
she was in the school. She said, "It's nice to have a chance to interact with other teachers outside 
of the regular school day." A third parent said that her son was having problems with math, and 
the teacher recommended that they go to Family Math. She feels that the teacher knows she is 
doing "everything I can" and it "clinches our relationship." 

How would you change Family Math to make it work better? The major complaint from three 
parents was that there should be longer Family Math workshops and more Family Math workshops. 
One parent said, "Just when I get into it, it's time to go." Several parents thought it should be more 
than three sessions. Several other parents said that they wished more parents would go, particularly 
those who were not involved in other activities in the school. They seemed to enjoy it so much that 
they wanted to share this pleasurable experience with others. 

Other Observations . Parents with limited English were asked how this affected their participation 
in the Family Math workshops. They were unanimous in saying that a bilingual leader or other 
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person fluent in their language was not necessary for them to enjoy and participate in the workshop. 
While several admitted that initially their reluctance to participate was primarily due to their limited 
English, they were clear that once they attended it was not an obstacle. They reported that the 
teachers were very friendly and made them feel welcome. 

Probably the most revealing information is that virtually all of the people interviewed said that they 
will be attending Family Math again in Spring 1994, and that they are looking forward to it. 

Summary . The response of this diverse group of parents to Family Math was very positive. The 
appeal of Family Math stemmed from the joint parent-child approach; parents felt comfortable 
bringing siblings as well so that they could all "have fun together." Motivation for attending was to 
learn more math and to develop an interest in math for their children. Changes in math-related 
interactions at home as well as in the children's attitudes toward math (improved) were reported. 
While participation in other school activities did not change, greater confidence in relating to 
teachers and other school personnel were reported. Parent enthusiasm is evident through their 
continued participation in Family Math, their desire to expand the program hours and number of 
sessions and their wish to "share" it with more parents. 

Impact 

Discussion 

The most important result of this evaluation is the finding that the students in the Spring 1994 who 
had prior attendance in FamiN Math workshops scored higher (national percentile) than those who 
had more limited or no Family Math experience. This suggests that more sessions of Family Math 
may lead to improved mathematics performance. It is unclear whether the additional math 
experiences directly affect performance or whether parent participation mediates student 
performance. Students may simply score higher because they have had more instruction, or they 
may score higher because parents now spend more time helping them with their homework or some 
combination of factors. Since students were not randomly assigned to groups, it is also possible that 
self-selection of the most highly motivated parents into the prior experience group provides the basis 
for these findings. A follow-up study where students are randomly assigned to groups will further 
clarify this issue. Further, the significant finding with regard to greater parent-school involvement 
suggests that participation in Family Math may result in greater school involvement which has been 
shown to be related to improved student performance. 

It is necessary to be cautious, however, \ dth results based on so few sessions. As was discussed 
above, few of the families attended all three ramily Math sessions each term. In fact, fewer than 
one quarter of the students attended all three sessions. This would severely limit the potential 
impact of the intervention and makes the two significant results more striking. The other 
nonsignificant results as well as the interview data suggest that three sessions is not enough to expect 
change and an expanded or ongoing program might be more effective. 

Attitude changes of parents on the questionnaires with regard to their own and their perception of 
their child's self-confidence with respect to mathematics were not significant. Yet, the 10 parents 
interviewed clearly expressed that both their attitudes and their behaviors had changed. This 
inconsistency is hard to explain and may reflect the impreciseness of the attitude measures. 
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It is clear from both the interview data and the parent evaluations that the quality of the sessions was 
outstanding and that the parents were vary satisfied and enthusiastic about the Family Math program 
and eager to continue their participation. 

In view of the several positive indications in this study, further research on the impact of Family 
Math should focus on extending the longitudinal nature of the experimental experience to encompass 
three or more years of Family Math attendance in successive semesters as well as random 
assignment into groups . In addition, the number of sessions in each j mester should be increased 
to 5 or 6 and incentive techniques devised to greatly improve the regularity of attendance. Another 
variation which might prove interesting, would be the introduction of Family Science into the 
sequence, particularly at the upper elementary and middle school grades to further augment interest 
in mathematics, science, engineering and technology. Family Science is a program, similar to 
Family Math., which uses hands-on science activities. 

Additional Outcomes 

There are now 69 qualified Family Math teachers including 5 staff developers in District #15 as of 
the end of this project. Of these, 47 directly participated in one or more phases of the project. In 
addition, 7 other District #15 teachers met with Virginia Thompson and Karen Mayfield on January 
28, 1994. Also, 78 teachers from 24 of the 31 CSD's, other than #15, plus another 7 entities either 
participated in the meeting with Virginia Thompson, received her new materials, or were newly 
qualified as teachers or trainers in Family Math. Thus, a total of 132 professionals were in some 
way directly affected by this project. We have seen from the teacher interviews that many will use 
elements c the Family Math materials to change what they do in their regular math classes and that 
the enthusiasm that Family Math generates among participants is also found among the teaching 
faculty. Since most elementary school teachers will work with 30 or more students each year and 
some work with many more as math specialists, or in middle schools where a math teacher may see 
several classes a day due to departmentalized scheduling, the impact will be felt by large numbers of 
students through these teachers. Furthermore, because of the normal flow of students through the 
grades, the effect is continuous on new groups each year. By having created four new staff 
developers, additional teachers of Family Math will be qualified as these staff developers train 
others to teach Family Math. 

As parents increasingly take a serious interest in their children *s growth in math and other critical 
subjects, it is likely that families will be better informed about their educational choices and the 
connection between careers and academics. Furthermore, as parents and children have fun working 
cooperatively in Family Math activities, intrafamily relationships can be expected to grow in positive 
ways. With more than 200 children in the experimental group, they probably represent more than 
170 families who have been directly exposed to Family Math in this project. Moreover, there will 
be expanding opportunities for family participation in Family Math now that all 21 elementary 
schools in District #15 have one or more qualified Family Math teachers. This is a goal which is 
most desirable to ensure access to this experience by all families who wish to participate but has not 
been achieved elsewhere in the City. 
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TABLE 1. BACKGROUND DATA FOR CONTROL & EXPERIMENTAL GROUPS 

1993 & 1994 COHORTS 



1993 

(N = 89) (N - 101) 



1994 

(N = 234) (N = 211) 



VARIABLE 


CONTROL 


EXPER. 


CONTROL 


EXPER. 


GRADE 










4 

5 
6 
7 


33.0 % 
50.5 % 
16.5 % 


39.2 % 
43.1 % 
17.6 % 


36.8 % 

38.5 % 

22.6 % 
2.1 % 


40.3 % 
40.8 % 
16.6 % 
2.4 % 


GENDER 










MALE 
FEMALE 


43.8 % 
56.2 % 


44.6 % 
55.4 % 


48.7 % 
51.3 % 


45.9 % 
54.1 % 


LEP' 


9.0 % 


16.8 % 


6.5 % 


7.1 % 


FREE LUNCH 


72.5 % 


68.1 % 


62.7 % 


51.3 % 


PRIOR EXP 2 


15.4 % 


48.0 % 


7.3 % 


29.6 % 


NSESSIONS 3 










A 

u 

1 

2 
3 
>3 


1 ftft ft % 


38.9 % 
36.8 % 
24.2 % 


inn n % 


31.1 % 
25.8 % 
32.5 % 
6.3 % 


ATTENDANCE BY AT 
LEAST ONE PARENT 










SESSION 1 
SESSION 2 
SESSION 3 




87.5 % 
62.5 % 
32.3 % 




86.3 % 
65.7 % 
49.0 % 



'LEP = Limited English Proficient 

2 PRIOR EXP = Any Prior Experience Attending Family. Math Programs 
3 NSESSIONS = Number of Sessions Attended This Semester 
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TABLE 2. PARENT EVALUATIONS OF FAMILY MATH TRAINING 

QUESTION 1923 1994 

AVERAGE RATING 1 OF OVERALL 1.4 (N = 66) 1.3 (N = 55) 

QUALITY OF THE SESSIONS 

AVERAGE RATING OF 1.4 (N = 66) 1.3 (N = 55) 

PRESENTATION OF MATERIALS 

AVERAGE RATING OF EVERYDAY 1.4 (N = 65) 1.5 (N = 55) 

USEFULNESS OF SESSIONS 

I'M ABLE TO HELP MY CHILD 

WITH MATH. % YES 83.3 (N = 66) 68.5 (N = 55) 

I HAVE MATH MATERIALS TO USE 

WITH MY CHILD AT HOME % YES 72.7 (N = 66) 74.1 (N = 54) 

I KNOW MORE ABOUT MATH % YES 72.7 (N = 66) 75.9 (N = 54) 

I'VE LEARNED HOW MATH IS 

TAUGHT IN SCHOOL % YES 80.3 (N = 66) 75.9 (N = 54) 

WOULD YOU LIKE TO ATTEND 
MORE FAMILY MATH CLASSES 

NEXT YEAR? % YES 91.0 (N = 67) 96.3 (N = 54 

WOULD YOU RECOMMEND 

FAMILY MATH TO A FRIEND? % YES 94.0 (N = 67) 96.4 (N = 55) 



1 Ratings were on a 1-5 Scale where 1 = EXCELLENT and 5 = POOR. 
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TABLE 3. 



ANALYSIS OF COVARIANCE COMPARING THE POST SCORES 
OF THE CONTROL & EXPERIMENTAL GROUPS 



1993 COHORT - ADJUSTED POST SCORE MEAN S 



GROUP 
CONTROL 



EXPERIMENTAL 
(No Prior Family 
Math Experience) 

EXPERIMENTAL 
(Prior Family 
Math Experience) 

STATIST SIGNIF 



GROUP 
CONTROL 



M 
A 
T 
H 



20.4 
(N = 89) 



20.1 
(N = 50) 



20.7 
(N = 45) 

NS 



N 
P 



65.7 
(N = 78) 



66.8 
(N = 42) 



73.0 
(N = 41) 

NS 



S 
T 
U 
A 
T 
T 

2.8 
(N = 88) 



2.9 
(N = 47) 



3.0 
(N = 44) 

NS 



P 
A 
R 
1 



3.0 
(N = 39) 



3.2 
(N = 20) 



2.8 
(N = 23) 

NS 



P 
A 
R 

2 



3.0 
(N = 39) 



3.2 
(N = 20) 



3.1 
(N = 23) 

NS 



1994 COHORT - ADJUSTED POST SCORE MEANS 



M 
A 
T 
H 



10.7 
(N = 190) 



N 
P 



69.5 
(N = 223) 



S 
T 
U 
A 
T 
T 

3.0 
(N = 170) 



P 
A 
R 
1 



3.1 
(N = 75) 



EXPERIMENTAL 

(No Prior Family 10.5 

Math Experience) (N = 104) 

EXPERIMENTAL 
(Prior Family 11.6 

Math Experience) (N = 55) 



66.6 2.9 3.0 

(N = 114) (N = 80) (N = 41) 



STATIST SIGNIF 



NS 



77.9 
(N = 51) 

SIG 



3.1 
(N = 42) 

NS 



3.1 
(N = 31) 

NS 



P 

A 
R 

2 



3.2 
(N = 75) 



3.2 
(N = 41) 



3.3 
(N = 31) 

NS 
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20 



p 

A 
R 

3 



2.9 
(N = 39) 



3.0 
(N = 20) 



2.9 
(N = 22) 

NS 



P 
A 
R 

3 



2.3 
(N = 74) 



2.8 
(N - 41) 



2.9 
(N = 31) 

SIG 



TABLE 3. (CONTINUED) 



MATH = Post performance-based assessment mathematics score 
NP = Percentile score on standardized mathematics examination 
STUATT = Average score on the 10 item Student Attitude Questionnaire 

(1 = Most negative attitude, 4 = Most positive attitude) 
PARI = Average score on items 1-7 of the Parent Questionnaire 

( 1 = Most negative attitude, 4 = Most positive attitude) 
PAR2 = Average score on items 8-11 of the Parent Questionnaire 

(1 = Most negative attitude, 4 = Most positive attitude) 
PAR3 = Average score on items 12-16 of the Parent Questionnaire 

(1 = Most negative attitude, 4 = Most positive attitude) 
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APPENDIX A 



Agenda and Presentation Outline for March 17, 1993 Meeting 
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OFFICE OF THE COMMUNITY SUPERINTENDENT 
COMMUNITY SCHOOL DISTRICT 15 
360 SMITH STREET 
BROOKLYN, NEW YORK 1 1231 

William P. Casey Loyda R. Alfalla 

Community Superintendent Deputy Superintendent 

March 8, 1993 



CASE/District 15 
Family Math Meeting 
March 17, 1993 
AGENDA 



- Welcome and Introductions 

- Overview of CASE/CUNY/District 15 

Mathematics Project: 

Dr. S. Brodsky, CASE/CUNY 

- Specific Administration Matters: 

- Portfolios: 

Pre-test 

Attitude Survey 

Spring '93 Test Score 

- Materials 

- Payroll Issues 

- Question?, Comments and Concerns 
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OUTLINE OF CSD#15--CASE/CUNY GRAD SCHOOL FAMILY MATH PROJECT 
FUNDED BY A GRANT FROM THE CHARLES A. DANA FOUNDATION 



Formal Title - An Urban FAMILY MATH Collaborative 



Project Directors 



Evaluators 



Josephine Urso for CSD #15 
Dr. Stan Brodsky for CASE/C'TOY Grad School 
Dr. Marian Fish, CUNY Graduate Ed Psych Faculty 
Dr. Alan Gross, CUNY Graduate Ed Psych Faculty 



Significance : 

We expect this project to be the definitive study of the effectiveness of Family Math and 
that it will be a national model. Results will be reported at national meetings and in the 
professional literature. 

Major Hypothesis : 

Participation in Family urograms in successive semesters will have a positive impact 

on family attitudes, certain student abilities and math-related student choices in middle schools. 

Additional Purposes : 

To field-test and evaluate newly developed Family Math materials for middle school 
grades. These materials are being developed by FAMILY MATH/EQUALS. at the Lawrence Hall 
of Science at the University of California, Berkeley. 

To prepare more persons to teach Family Math programs in CSD #15 and elsewhere in 
NYC. Dates for a training session will be announced — probably scheduled for July 1993. 

* Recruitment : 

We want as many families as possible (in the grades to be designated) to make a 
commitment to attend all sessions (3) of a Family Math program in this semester (Spring '93) at 
their home school AND attend all sessions of Family Math programs in Fall '93 (3 sessions) and 
Spring '94 (3 sessions) given in the school that the children are attending at the time of the 
programs. We know that some will make the commitment but might miss some sessions and 
they will still be part of the study - but we want as many full attenders as possible to really 
help the study to show what Family Math can do. Shoot for 20 or more families with both 
grades represented. Set up portfolio for each FM & non-FM student in the study. 

Documents For The Study : 

Just before this semester's program - 

* Student Pre-Test for ail Family Math students. 

* Student Attitude Questionnaire Form 1 to all Family Math students AND at least 
as many non-Family Math students from the same grades and classes. 

* Parent Attitude Questionnaire to all Family Math parents AND to the parents of 
the non-Family Math students included above. 

* Student Background Questionnaire to be completed by the Family Math teachers 
for all Family Math students and non-Family Math students included above. 

After the semester's program — 

♦Student Post-Test for all Family Math students. 

* Student Attitude Questionnaire Form 2 as above for all FM and non-FM students. 

* Repeat Parent Attitude Questionnaire for all FM parents as above. 

* Parent Family Math Evaluation Form to all Family Math parents who attended. 
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APPENDIX B 



Evaluation Forms 

Student Background Questionnaire 
Student Attitude Questionnaire 
Parent Questionnaire (English Version) 
Parent Questionnaire (Spanish Version) 
Parent Family Math Evaluation Form 
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Semester: 

Term Year 

STUDENT BACKGROUND QUESTIONNAIRE 

V * 

V * 

1. Student Name: : : 

2. OSIS Number: 3. Date of Birth:_ = - 

Mo, - Bay - Year 

4. Class: 5. School: : 

6- Gender (M = Male, F = Female): 

7. Is this student designated as Limited English Proficient? (Y = Yes, N = No): 

8. Does this student receive Free or Reduced Price Lunch? (Y = Yes, N = No): 

9. .V Has this student attended any previous Family Math 

. program? (Y = Yes, N = No, U = Unknown): 

10. Is this student enrolled in a Family Math program 

this semester? (Y = Yes, N = No): - 

11 # At the end of this semester, please record the number 

of Family Math sessions the student attended: ■ 

12. For each session attended, please record which parent(s) 

or other adult attended with the student. 

Mother Father Both Other Person 

Session 1 

Session 2 

"Session 3 mmmm ~ ~ " ' 

13. Pre-Test Score on Mathematics Performance Test: 

14. Post-Test Score on Mathematics Performance Test: 

15. CAT - 5 National Percentile Score: 

16. Next Semester's School: 

17. Next Semester's Grade: 

Many thanks for til your efforts in support of thia project* Your contribution ts important and ts much appreciated. 



Student Name 



STUDENT ATTITUDE QUESTIONNAIRE 

% 



STUDENT OS IS # 



Date of Administration 



TELL US WHAT YOU THINK BY PUTTING A CIRCLE AROUND THE NUMBER WHICH BEST 
DESCRIBES YOURSELF. REMEMBER, THERE ARE NO WRONG OR RIGHT ANSWERS. WE 
JUST WANT TO KNOW WHAT YOU THINK. 



1. I enjoy doing math. 

2. A math test would scare me. 

3. I learn math easily. 

4. I do well in math. 

5. Math is a difficult 
subject for me to learn 

6. I like math puzzles or 
games that use math. 

7. I use math quite a lot. 

3. I like doing math homework. 

9 . I have more trouble with math 
than with other subjects. 

10. I talk about math with my 
mother or father. 



NEVER 
l 
1 
l 
l 



1 
1 
1 



SOME OP 
THE TIME 

2 

2 

2 

2 



2 
2 
2 



MOST OP 
THE TIME 

3 

3 

3 

3 



3 
3 
3 



ALWAYS 
4 
4 
4 
4 



4 
4 
4 



Form 1 
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PARENT 



QUESTIONNAIRE 



Your Child's Name: 



Child's OSIS No. : 



Your Child's Grade: 



Date This Form Completed:. 



mo. -day -yr 

Here are some things that some people might say about math, or about learning math. We Would like to 
know your reactions to them. Please indicate how you feel about these statements by circling SD if 
you STRONGLY DISAGREE, or D if you MILDLY DISAGREE, or A if you MILDLY AGREE, 
or SA if you STRONGLY AGREE. Remember, there are no right or wrong answers. We want to know 
what you think. 

STRONGLY MILDLY : MILDLY STRONGLY 
DISAGREE DISAGREE AGREE AGREE 



1 . I enjoy doing math. 


SD 


D 


A 


SA 


2. A math test would scare me. 


SD 


D 


A 


SA 


3. I learn math easily. 


SD 


D 


A 


SA 


H-. main IS a QITTICUIl SUUjcCl IO Icdlll. 


SD 


D 


A 


SA 


5. Math is one of my favorite subjects. 


SD 


D 


A 


SA 


6. i like math puzzles or games that use math. 


SD 


D 


A 


SA 


7. 1 use math quite a lot from day to day. 


SD 


D 


A 


SA 


8. My child seems to like math a lot. 


SD 


D 


A 


SA 


9. 1 regularly supervise my child's 
math homework. 


SD 


D 


A 


SA 


10. I point out to my child how math is used 
in our day-to-day life. 


SD 


D 


A 


SA 


1 1 . My child is doing well in math in school. 


SD 


D 


A 


SA 



The following are some different statements about parent/school relationships. Please indicate how you 
feel about these statements by circling SD if you STRONGLY DISAGREE, or D if you MILDLY DISAGREE, 
or A if you MILDLY AGREE, or SA if you STRONGLY AGREE. Remember, there are no right or wrong 
answers. We want to know what you think. 



12. My child's school actively encourages 

parents to be involved. 

13. I generally don't talk with my child's teacher 

unless there is a problem. 

14. I regularly attend school events. 



SD 


D 


A 


SA 


SD 


D 


A 


SA 


SD 


D 


A 


SA 


SD 


D 


A 


SA 



16. I help in planning activities at my child's 

school. SD D A SA 



THANK YOU FOR YOUR COOPERATION IN COMPLETING THIS QUESTIONNAIRE. 
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CUESTIONARIO PARA PADRES Y MADRES 

Nombre de su hija/o: Numero de OSIS:^ 

Grado de su hija/o: Fecha: - - 

mes-dia-anb 

Estas son algunas de las cosas que la gents diccacerca de matematicas o del aprendizaje de 
matematicas. Nos gustarfa saberqu^ piensa de estas opiniones. Por favor indique si estd'de acuerdo 
con estas opiniones. Ponga un circulo airededor de CD si usted esta' COMPLETAMENTE EN 
DESACUERDO, o un circulo airededor de D si usted esta MAS 0 MENOS EN DESACUERDO, o A si esta" 
MA'S 0 MENOS ACUERDO, o CA si esta COMPLETAMENTE DE ACUERDO. Recuerdo, no hay 
respuestas correctas o inccrrectas. Solo queremos saber lo que usted piensa. 

COMPLETAMENTE MAS 0 MENOS mXs 0 MENOS COMPLETAMENTE 



EN DESACUERDO 


EN DESACUERDO 


DE ACUERDO 


DE ACUERDO 


1 . Disfruto estudiando matematicas. 


CD 


D 




A 


CA 


2. Un exa"men de matematicas me asusta. 


CD 


D • 




A 


CA 


3. Aprendo matematicas facilmente. 


CD 


D 




A 


CA 


4. Matematicas es i<n materia diffcil de 
aprender. 


CD 


D 




A 


CA 


5. Matematicas es una de mis materias 
preferidas. 


CD 


D 




A 


CA 


6. Me gustan los rompecabezas de matematicas 
o los juegos que usan matematicas. CD 


D 




A 


CA 


7. Uso matematicas bastante cada dia. 


CD 


D 




A 


CA 


8. At parecer a mi hija/o le gustan mucho 
las matema'ticas. 


CD 


D 




A 


CA 


9. Supervise reguiarmente la tarea de 
matematicas de mi hija/o. 


CD 


D 




A 


CA 


10. Le indico a mi hija/o como se usan las 
matematicas en la vida diaria. 


CD 


D 




A 


CA 


11. A mi hija/o le va bien en matematicas 
en la escuela. 


CD 


D 




A 


CA 


Las siguientes son algunas de las opiniones sobre las relaciones entre los padres de familia y la escuela. 
Po favor indique si esta de acuerdo con estas opiniones. Ponga un circulo airededor de CD si usted esta' 
COMPLETAMENTE EN DESACUERDO, o un circulo airededor de D si usted esta MAS 0 MENOS EM 
DESACUERDO, o A si estzf MA'S 0 MENOS ACUERDO, o CA si esta'COMPLETAMENTE DE ACUERDO. 
Recuerde, no hay respuestas correctas o incorrectas. Solo queremos saber lo que usted piensa. 


12. La escuela de,mi hija/o anima activamente 
la participacidn de los padres. 


CD 


D 


A 




CA 


13. Generalmente no hablo con el 
maestro o la maestra de mi hija/o 
a menos que haya un problema. 


CD 


D 


A 




CA 


14. Reguiarmente voy a eventos escolares. 


CD 


D 


A 




CA 


15. Soy voluntaria/o reguiarmente en la 
escuela de mi hija/o. 


CD 


D 


A 




CA 


16. Ayudo a planear actividades en la 
escuela de mi hija/o. 


CD 


D 


A 




CA 
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PARENT FAMILY HATH EVALUATION FORM 



Today's Date:_ 



1 . Student's Name: 



Student's OSIS No.: 



2. Please rate the Family Math Sessions on each of the following areas. Circle the number 
which shows your answer. 



(a) Overall Quality of the Sessions 

(b) Presentation of Materials 

(c) Everyday Usefulness of Sessions 



Excellent 
1 

1 
1 



2 
2 
2 



3 
3 
3 



Poor 
4 5 
4 5 
4 5 



3. After going to the Family Math sessions, I fee! that I have accomplished the following: 
Check all items that apply to you. 

I'm able to help my child with math. 

I have math materials to use with my child at home. 

I know more about math. 



I've learned how math is taught in school. 



(Circle Your Answer) 

4. Would you like to attend more Family Math classes next year? Yes No Not Sure 



5. Would you recommend Family Math to a friend? 



Yes 



No 



Not Sure 



Thank you for your participation and cooperation. 



25 



30 



ERIC 



